Method and System for Customizing User Experience

ABSTRACT

Embodiments disclosed herein generally related to a method and system for customizing a customer experience. In one embodiment, a method is provided herein. A computing system receives from a computing device positioned in a facility one or more video streams. The one or more video streams capture a customer in the facility. The computing system identifying an identity of the customer by parsing the one or more video streams to identify one or more audio or visual cues of the customer. The computing system determines, based on the identity of the customer, that the customer has one or more previous transactions at the facility. The computing system predicts, based on the one or more previous transactions, a new transaction at the facility. The computing system notifies the computing device positioned in the facility in preparation of the new transaction.

CROSS-REFERENCE TO RELATED APPLICATION INFORMATION

This is a continuation of U.S. patent application Ser. No. 16/256,617,filed Jan. 24, 2019, which is incorporated herein by reference in itsentirety.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to a method and a system forcustomizing a user experience.

BACKGROUND

When customers visit their frequently visited facilities, often thosecustomers transact similarly to past transactions that each facility.For example, when a customer visits a particular local restaurant, thecustomer may frequently order one or two items each time. During busyhours for the facility, for example, frequent customers may becomefrustrated by long lines and preparation times, despite being adedicated customer to the facility. Further, because the customer hastypically performed the same transaction at this particular facility,once a customer enters the facility, it should be easy for the facilityto recognize the customer and the customer's “usual” order. However, dueto the turn-over rate at such facilities, as well as the movement awayfrom human employees for electronic point-of-sale terminals, thehistorical concept of a “usual customer” is quickly being eliminated asa result of unfamiliarity with staff and impersonalization oftechnology.

SUMMARY

In one embodiment, a method of customizing a customer experience isdisclosed herein. A computing system receives from computing devicepositioned in a facility one or more first video streams captured by afirst camera positioned at an entrance of the facility. The one or morefirst video streams capture a customer in the facility. The computingsystem identifies an identity of the customer by parsing the one or morefirst video streams to identify one or more audio or visual cues of thecustomer. The computing system determines, based on the identity of thecustomer, that the customer has one or more previous transactions at thefacility. The computing system analyzes the one or more previoustransactions at the facility to identify a transaction pattern at thefacility. The computing system predicts, based on the identifiedtransaction pattern in the one or more previous transactions, a newtransaction at the facility. The computing system receives, from thecomputing device positioned in the facility, one or more second videostreams captured by a second camera positioned at a point-of-saleterminal in the facility. The computing system parses the one or moresecond video streams to confirm that the customer remains in thefacility. The computing system notifies the computing device positionedin the facility in preparation of the new transaction.

In some embodiments, the computing system further determines that thecustomer has pre-authorized payment from a customer account. Thecomputing system notifies the computing device positioned in thefacility of the pre-authorized payment.

In some embodiments, analyzing the one or more previous transactions atthe facility to identify a transaction pattern at the facility includes,the computing system identifying a day of the week and time of day eachthe previous transaction occurred, for each previous transaction of theone or more previous transactions at the facility.

In some embodiments, wherein predicting, based on the one or moreprevious transactions, a new transaction at the facility includes thecomputing system identifying a current day of the week. The computingsystem identifies a current time of the current day. The computingsystem generates a prediction based on the identified transactionpattern in one or more previous transactions, the current day of theweek, and the current time of the current day.

In some embodiments, the computing system further receives aconfirmation from the customer in response to the customer viewing aconfirmation message at a point-of-sale terminal. The computing systemtransmits the confirmation to the computing device positioned in thefacility.

In some embodiments, identifying the identity of the customer by parsingthe one or more video streams to identify one or more audio or visualcues of the customer includes the computing system analyzing one or morefacial features of the customer. The computing system determines theidentity of the customer based on the one or more facial features.

A computing system receives from a computing device positioned in afacility one or more video streams. The one or more video streamscapture a customer in the facility. The computing system identifying anidentity of the customer by parsing the one or more video streams toidentify one or more audio or visual cues of the customer. The computingsystem determines, based on the identity of the customer, that thecustomer has one or more previous transactions at the facility. Thecomputing system predicts, based on the one or more previoustransactions, a new transaction at the facility. The computing systemnotifies the computing device positioned in the facility in preparationof the new transaction.

In some embodiments, the computing system further determines determiningthat the customer has pre-authorized payment from a customer account.The computing system notifies notifying the computing device positionedin the facility of the pre-authorized payment.

In some embodiments, predicting, based on the one or more previoustransactions, a new transaction at the facility includes the computingsystem identifying a current day of the week. The computing systemidentifies a current time of the current day. The computing systemgenerates a prediction based on the one or more previous transactions,the current day of the week, and the current time of the current day.

In some embodiments, receiving, from the computing device positioned inthe facility, one or more video streams includes the computing systemreceiving a first video stream from a first camera positioned at anentrance of the facility.

In some embodiments, the computing system further receives a secondvideo stream from a second camera positioned at a point-of-sale terminalin the facility.

In some embodiments, the computing system further receives aconfirmation from the customer in response to the customer viewing aconfirmation message at a point-of-sale terminal. The computing systemtransmits the confirmation to the computing device positioned in thefacility.

In some embodiments, identifying the identity of the customer by parsingthe one or more video streams to identify one or more audio or visualcues of the customer includes the computing system analyzing one or morefacial features of the customer. The computing system determines theidentity of the customer based on the one or more facial features.

In another embodiment, a system is disclosed herein. The system includesa processor and a memory. The memory has programming instructions storedthereon, which, when executed by the processor, performs an operation.The operation includes receiving, from a computing device positioned ina facility, one or more video streams, the one or more video streamscapturing a customer in the facility. The operation further includesidentifying an identity of the customer by parsing the one or more videostreams to identify one or more audio or visual cues of the customer.The operation further includes determining, based on the identity of thecustomer, that the customer has one or more previous transactions at thefacility. The operation further includes predicting, based on the one ormore previous transactions, a new transaction at the facility. Theoperation further includes notifying the computing device positioned inthe facility in preparation of the new transaction.

In some embodiments, the operation further includes determining that thecustomer has pre-authorized payment from a customer account. Theoperation further includes notifying the computing device positioned inthe facility of the pre-authorized payment.

In some embodiments, predicting, based on the one or more previoustransactions, a new transaction at the facility includes identifying acurrent day of the week, identifying a current time of the current day,and generating a prediction based on the one or more previoustransactions, the current day of the week, and the current time of thecurrent day.

In some embodiments, receiving, from the computing device positioned inthe facility, one or more video streams, the one or more video streamscapturing the customer in the facility includes receiving a first videostream from a first camera positioned at an entrance of the facility.

In some embodiments, the operation further includes receiving a secondvideo stream from a second camera positioned at a point-of-sale terminalin the facility.

In some embodiments, the operation further includes receiving aconfirmation from the customer in response to the customer viewing aconfirmation message at a point-of-sale terminal. The operation furtherincludes transmitting the confirmation to the computing devicepositioned in the facility.

In some embodiments, identifying the identity of the customer by parsingthe one or more video streams to identify one or more audio or visualcues of the customer includes analyzing one or more facial features ofthe customer and determining the identity of the customer based on theone or more facial features.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentdisclosure can be understood in detail, a more particular description ofthe disclosure, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrated onlytypical embodiments of this disclosure and are therefore not to beconsidered limiting of its scope, for the disclosure may admit to otherequally effective embodiments.

FIG. 1 is a block diagram illustrating a computing environment,according to one exemplary embodiment.

FIG. 2 is a block diagram illustrating one or more computing componentspositioned in a facility, according to one exemplary embodiment.

FIG. 3 is a flow diagram illustrating a method of customizing a customerexperience, according to one exemplary embodiment.

FIG. 4 is a block diagram illustrating a computing environment,according to one embodiment.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures. It is contemplated that elements disclosed in oneembodiment may be beneficially utilized on other embodiments withoutspecific recitation.

DETAILED DESCRIPTION

One or more techniques disclosed herein are generally directed to amethod and a system for customizing customer experiences. When acustomer walks into a facility (e.g., local shop, restaurant, etc.),customers typically make similar or the same transaction at the facilityas they previously made. For example, when a customer walks into aparticular restaurant, the customer is likely to order either “harvestbowl” or “guacamole greens.”

The one or more techniques disclosed herein are directed to customizinga customer experience at a facility. In particular, the one or moretechniques disclosed herein generally relate to a computing system thattracks a customer's transaction history at one or more facilities, andgenerates a predicted transaction for the customer upon detecting thecustomer arriving at or within a given facility. For example, the one ormore techniques disclosed herein leverage one or more cameras to trackcustomer activity in a facility, as well as one or more point-of-saleterminals in the facility, to generate a “cookie” for the customer thatcreates a record of each transaction at each facility. Based on thecustomer's transaction history (i.e., collection of cookies), the systemdisclosed herein is able to predict a customer's transaction upondetecting a customer at a particular facility. For example, the systemmay leverage the one or more cameras and facial recognition software toidentify when the customer arrives (or is inside) the facility.

By accurately predicting a customer's transaction based on thecustomer's transaction history, the system can communicate with thefacility, such that the facility may begin preparation of the customer'stransaction. Such anticipatory operation helps improve the customerthroughput at the facility, and therefore the profitability of thefacility.

Such embodiments are not to be limited to food service facilities. Suchexemplary facilities may include, but are not limited to, movie theatres(e.g., the present system may be configured to suggest movies based onuser experience and time), clothing stores (e.g., the present system maybe configured to suggest clothes based on size and prior orders), bookstores (e.g., the present system may be configured to suggest booksbased on past user preferences), and similar facilities through whichthe present system can leverage past

FIG. 1 is a block diagram illustrating a computing environment 100,according to one embodiment. Computing environment 100 may include atleast a facility 102 and an organization computing system 104communicating via network 105. In some embodiments, computingenvironment 100 may further include a client device 152 and a facilityweb server 154 communicating with facility 102 and organizationcomputing system 104 via network 105.

Network 105 may be of any suitable type, including individualconnections via the Internet, such as cellular or Wi-Fi networks. Insome embodiments, network 105 may connect terminals, services, andmobile devices using direct connections, such as radio frequencyidentification (RFID), near-field communication (NFC), Bluetooth™,low-energy Bluetooth™(BLE), Wi-Fi™, ZigBee™, ambient backscattercommunication (ABC) protocols, USB, WAN, or LAN. Because the informationtransmitted may be personal or confidential, security concerns maydictate one or more of these types of connection be encrypted orotherwise secured. In some embodiments, however, the information beingtransmitted may be less personal, and therefore, the network connectionsmay be selected for convenience over security.

Network 105 may include any type of computer networking arrangement usedto exchange data. For example, network 105 may be the Internet, aprivate data network, virtual private network using a public networkand/or other suitable connection(s) that enables components in computingenvironment 100 to send and receiving information between the componentsof system 100.

Facility 102 may be representative of a business or organizationassociated with organization computing system 104. Exemplary businessesor organizations may include, but are not limited to, restaurants,coffee shops, donut shops, gas stations, and the like. Facility 102 mayinclude one or more point-of-sale (POS) terminals 110, one or morecameras 112, and a controller 114. POS terminals 110 may be configuredto interface with a customer to facilitate the ordering and payment ofgoods and/or services. In some embodiments, each POS terminal 110 may beconfigured to interface with a payment device of a customer. Suchpayment devices may include, a mobile device, a credit card, a debitcard, a gift card, and the like. In some embodiments, each POS terminal110 may be configured to receive one or more banknotes as a form ofpayment.

In some embodiments, one or more POS terminals 110 may be operated by anemployee of facility 102. For example, employee at a POS terminal 110may enter a customer's transaction and facilitate payment between thecustomer and the organization via POS terminal 110. In some embodiments,one or more POS terminals 110 may be operated by the customer. Forexample, the customer may be provided with a “self-serve” option, inwhich the customer can enter his or her transaction via a display deviceand provide payment via POS terminal 110, without any interaction fromthe employee.

Each POS terminal 110 may be representative of a general purposecomputing device programmed to receive transactions from one or morecustomers and facilitate payment between the one or more customers andfacility 102.

One or more cameras 112 may be configured to monitor one or morecustomers throughout facility 102. One or more cameras 112 may beconfigured to monitor one or more customers as each customer enters andexits facility 102. In some embodiments, one or more cameras 112 may beconfigured to track a duration in which one or more customers atfacility 102. In some embodiments, one or more cameras 112 may beconfigured to monitor customer transaction habits. Generally, one ormore cameras 112 may be configured to track customer movement and habitsthroughout facility 102.

Controller 114 may be configured to manage one or more POS terminals 110and one or more cameras 112. For example, controller 114 may beconfigured to communicate with each POS terminal 110 and each camera 112via a local network (not shown). Local network may be substantiallysimilar to network 105. Controller 114 may be configured to aggregateone or more streams of data captured by one or more cameras 112.Controller 114 may further be configured to receive one or more sets oftransaction information transmitted by POS terminals 110. As such,controller 114 may be configured to collect and/or aggregate customerinformation captured by one or more cameras 112 and one or more POSterminals 110.

Controller 114 may be configured to communicate with organizationcomputing system 104 via network 105. For example, controller 114 mayinclude an application 115 executing thereon that facilitatescommunication between controller 114 and organization computing system104. Application 115 may be representative of a web browser that allowsaccess to a website or a stand-alone application. Controller 114 mayaccess application 115 to access functionality of organization computingsystem 104. Controller 114 may communicate over network 105 with webclient application server 114 of organization computing system 104. Forexample, client device 102 may be configured to execute application 115to access one or more functionalities of organization computing system104.

Organization computing system 104 may include at least handler 118 andtransaction agent 120. Both handler 118 and transaction agent 120 may becomprised of one or more software modules. The one or more softwaremodules may be collections of code or instructions stored on a media(e.g., memory of organization computing system 104) that represent aseries of machine instructions (e.g., program code) that implements oneor more algorithmic steps. Such machine instructions may be the actualcomputer code the processor of organization computing system 104interprets to implement the instructions or, alternatively, may be ahigher level of coding of the instructions that is interpreted to obtainthe actual computer code. The one or more software modules may alsoinclude one or more hardware components. One or more aspects of anexample algorithm may be performed by the hardware components (e.g.,circuitry) itself, rather as a result of an instructions.

Handler 118 may be configured to manage information stored in database106. For example, handler 118 may be configured to update clientinformation in database 106 when prompted by a customer, when promptedby employees of the facility, or when prompted by another component oforganization computing system 104.

Transaction agent 120 may be configured to predict a customertransaction prior to a customer arriving at, for example, a POS terminal110. Transaction agent 120 may include a facial analysis agent 122, avoice-to-text agent 124, and a machine learning module 125. Facialanalysis agent 122 may be configured to identify one or more users (orcustomers) using one or more video streams received from one or morecameras 112 positioned therein. Facial analysis agent 122 may beconfigured to parse through one or more video streams in real-time, toidentify one or more customers arriving at (or already inside) facility102, based on previously identified facial features stored in database106. For example, in some embodiments, facial analysis agent 122 maymatch a customer's face to a customer in database 106 using threedimensional facial recognition methodologies. In another example, facialanalysis agent 122 may match a customer's face to a customer in database106 using a skin texture analysis methodology. In another example,facial analysis agent 122 may match a customer's face to a customer indatabase 106 using a combination of three-dimensional facial recognitionand skin texture analysis.

Voice-to-text agent 124 may be configured to translate audibleutterances of a customer to text for further analysis by transactionagent 120. In some embodiments, each POS terminal 110 may include amicrophone to capture communications between a customer and an employeeoperation POS terminal 110. In some embodiments, cameras 112 may beconfigured to monitor POS terminals 110, such that cameras 112 maycapture communications between a customer and an employee (or anothercustomer) at facility 102. Voice-to-text agent 124 may include softwareconfigured to provide text-based descriptions of audible input.Transaction agent 120 may parse the text-based description to identifyone or more details of a customer's transaction at facility 102.

Machine learning module 125 may include one or more computer systemsconfigured to train a prediction model used by transaction agent 120 topredict details of a customer's future transaction. To train theprediction model, machine learning module 125 may receive, as input, oneor more streams of customer activity. The one or more streams ofcustomer activity may correspond to previous transaction at givenfacilities. Such streams of activity may include the facility, the timeof day, the day of the week, items purchased during the transaction, andthe like. In some embodiments, machine learning module 125 may furtherreceiver, as input, one or more streams of activity associated withadditional customers. As such, machine learning module 125 may leverageboth customer specific and customer agnostic information to predictdetails of a customer's transaction when, for example, the customerarrives at facility 102. Machine learning module 125 may implement oneor more machine learning algorithms to train the prediction model. Forexample, machine learning module 125 may use one or more of a decisiontree learning model, association rule learning model, artificial neuralnetwork model, deep learning model, inductive logic programming model,support vector machine model, clustering mode, Bayesian network model,reinforcement learning model, representational learning model,similarity and metric learning model, rule based machine learning model,and the like.

Organization computing system 104 may be in communication with database106. Database 106 may be configured to store information associated withone or more customers. For example, handler 118 may be configured togenerate one or more customer profiles 126, for each customer thataccesses a given facility 102. Customer profiles 126 may include facialfeatures 128, facilities 130, and account(s) 132.

Facial features 128 may correspond to those features identified byfacial analysis agent 122 as distinguishing features of a customer'sface. Accordingly, when organization computing system 104 receives oneor more streams of customer activity in facility 102, facial analysisagent 122 may query database 106 to compare facial features identifiedin the one or more streams of customer activity to one or more storedfacial features 128.

Facilities 130 may include a list of facilities 102 visited by aparticular customer. Each facility 130 may include one or moretransactions 134. Each transaction 134 may include one or more detailsof a particular transaction at facility 130. Such details may include,but are not limited to, day of the week, time of day, items purchasedduring the transaction, services rendered, means of payment, and thelike.

Accounts 132 may include one or more financial accounts linked to acustomer profile 126. For example, a customer may have the ability topre-approve automatic payment through by transaction agent 120, whentransaction agent 120 successfully identifies that customer has enteredfacility 102. In some embodiments, a customer may have the ability topre-approve payment from account 132, prior to transaction agent 120submitting a payment transfer request from a financial institutionassociated with account 132. Such accounts 132 may include, but are notlimited to, checking account, pre-paid account, savings account, and thelike.

Client device 152 may be operated by a customer (or user). For example,client device 102 may be a mobile device, a tablet, a desktop computer,or any computing system having the capabilities described herein. Clientdevice 152 may belong to or be provided to a customer or may beborrowed, rented, or shared. Customers may include individuals such as,for example, subscribers, clients, prospective clients, or customers ofan entity associated with organization computing system 104, such asindividuals who have obtained, will obtain, or may obtain a product,service, or consultation from an entity associated with organizationcomputing system 104.

Client device 152 may include at least application 155 executing thereonthat facilitates communication between client device 152 and one or morefacility web servers 154. Application 155 may be representative of a webbrowser that allows access to a website or a stand-alone application.Client device 152 may access application 155 to access functionality offacility web servers 154. For example, client device 152 may beconfigured to execute application 155 to access one or morefunctionalities of one or more facility web servers 154.

Each facility web server 154 may correspond to a respective facility102. Each facility web server 154 may be configured to manage one ormore web sites associated therewith. For example, assuming that facility102 is a movie theatre, web server 154 may host a movie theatre web site156 through which customers can buy tickets or access a customerprofiler managed by web server 154.

In some embodiments, a particular customer may be part of a loyaltyprogram associated with a particular facility 102. Through the loyaltyprogram, the customer may receive one or more updates via application155 executing on client device 152. For example, facility web server 154associated with facility 102 may prompt application 155 to push anotification to the customer, prompting the customer to confirm whetherthe customer is indeed within facility 102. In another example, facilityweb server 154 associated with facility 102 may prompt application 155to push a notification to the customer, prompting the customer toconfirm whether the predicted transaction (generated by transactionagent 120) is correct.

Further, in some embodiments, application 155 may allow customer to viewand edit past transactions at a particular facility 102. For example,via application 155, client device 152 may interface with facility'swebsite 156 to provide one or more transaction preferences. Web site 156may, in turn, interface with organization computing system 104 to updatea customer profile 126 with the one or more transaction preferences,accordingly.

Still further, in some embodiments, client device 152 may include ageolocation module 158 executing thereon. Geolocation module 158 may beconfigured to track a current location of client device 152. In someembodiments, organization computing system 104 may leverage geolocationmodule 158 to confirm whether the customer is indeed within or nearfacility 102. For example, client device 152 may enable locationservices for web site 156 (or application 155), such that facility webserver 154 may leverage geolocation module 158 to identify a currentlocation of the customer. Facility web server 154 may, in turn, relaythe location information to organization computing system 104 forfurther analysis.

FIG. 2 is a block diagram 200 illustrating an exemplary facility 102,according to one embodiment. As illustrated, facility 102 may have anentrance 202. In some embodiments, one or more cameras 112 may bepositioned exterior to facility 102. For example, camera 112 _(o) may bepositioned exterior to facility 102 and positioned to face entrance 202.Accordingly, camera 112 _(o) may be configured to capture one or morestreams of customer activity, as customers enter facility 102. As such,transaction agent 120 may be configured to predict a transaction of acustomer as the customer enters facility 102, so that transaction agent120 may communicate this predicted transaction to staff at facility 102.Accordingly, in some embodiments, the staff at facility 102 may beingpreparing the customer's transaction for a potential transaction.

As illustrated further, one or more computing components of computingenvironment 100 may be positioned within facility 102. Although FIG. 2illustrates each of the foregoing components within facility 102, thoseskilled in the art may readily understand that one or more componentsmay be positioned external to facility 102.

As shown, facility 102 may include POS terminal 110 ₁, 110 ₂, and 110_(n) Each POS terminal 110 may correspond to a particular camera 112.For example, camera 112 ₁ may be configured to capture customer activityat POS terminal 110 ₁, camera 112 ₂ may be configured to capturecustomer activity at POS terminal 110 ₂, and camera 112 _(n) may beconfigured to capture customer activity at POS terminal 110 _(n). Insome embodiments, facility 102 may include one or more cameras 112positioned to capture customer activity at a location separate from POSterminals 110. For example, as illustrated, camera 112 ₃ may bepositioned adjacent entrance 202.

Each camera 112 and POS terminal 110 may communicate with controller 114via a local network 205. Local network 205 may be of any suitable type,including individual connections via the Internet, such as cellular orWi-Fi networks. In some embodiments, local network 205 may connectterminals, services, and mobile devices using direct connections, suchas radio frequency identification (RFID), near-field communication(NFC), BlueTooth™, low-energy BlueTooth™ (BLE), Wi-Fi™, ZigBee™, ambientbackscatter communication (ABC) protocols, USB, WAN, or LAN. Because theinformation transmitted may be personal or confidential, securityconcerns may dictate one or more of these types of connection beencrypted or otherwise secured. In some embodiments, however, theinformation being transmitted may be less personal, and therefore, thenetwork connections may be selected for convenience over security.

Local network 205 may include any type of computer networkingarrangement used to exchange data. For example, local network 205 may bethe Internet, a private data network, virtual private network using apublic network and/or other suitable connection(s) that enablescomponents in and around facility 102 to send and receive informationbetween the components of facility 102.

FIG. 3 is a flow diagram illustrating a method 300 of customizing acustomer experience, according to one exemplary embodiment. Method 300may begin at step 302.

At step 302, organization computing system 104 may receive one or morestreams of customer activity from facility 102. In some embodiments,organization computing system 104 may receive one or more streams ofcustomer activity from controller 114. For example, controller 114 mayreceive one or more streams of customer activity via one or more cameras112 positioned in or proximate to facility 102. In some embodiments,organization computing system 104 may receive one or more streams ofcustomer activity directly from one or more cameras 112 positioned in orproximate to facility 102.

At step 304, organization computing system 104 may parse one or morevideo streams to identify a customer. For example, facial analysis agent122 may parse one or more video streams to identify one or more facialfeatures of each customer. Facial analysis agent 122 may query database106 to determine whether any of the identified one or more facialfeatures of a given customer match one or more facial features 128stored in database 128. In some embodiments, facial analysis agent 122may determine that a match does not exist. In these embodiments, facialanalysis agent 122 may store the identified facial features 128 indatabase 106 to create a new customer account 128. In some embodiments,facial analysis agent 122 may determine that a match does indeed exist.Identifying a match with facial features 128 may reveal an identity ofthe given customer.

At step 306, organization computing system 104 may identify an accountassociated with the identified customer. For example, organizationcomputing system 104 may backtrack from the identified facial features128 to identify a customer account 126 associated therewith.

At step 308, organization computing system 104 may determine whether thecustomer is a returning customer to facility 102. For example, handler118 may query database to retrieve all facilities 130 stored in database106 and associated with customer account 126. Determining whether thecustomer is a returning customer, aids in predicting the customer'stransaction at facility 102.

If, at step 308, organization computing system 104 determines that thecustomer is not a returning customer, the method 300 ends. If, however,at step 308, organization computing system 104 determines that thecustomer is a repeat customer, then method 300 proceeds to step 310.

At step 310, organization computing system 104 may access a transactionhistory of the customer associated with facility 102. For example,handler 118 may query database 106 to retrieve transaction information134 stored thereon. Transaction information 134 may have been previouslyadded to database 106 via handler 118 upon identifying a customertransaction via voice-to-text agent 124 or via one or more POS terminals110. Transaction information 134 may include a date of each transaction,a time of each transaction, goods transferred during the transaction(e.g., food, movie theatre tickets, clothing, books, etc.), servicesrendered during the transaction (e.g., car wash, massage therapist,chiropractor, etc.), and the like.

At step 312, organization computing system 104 may predict a transactionbased on at least transaction information 134. In some embodiments,handler 118 may provide as input to machine learning module 125 one ormore items of transaction information 134. For example, the one or moreitems of transaction information 134 may include a date of eachtransaction, a time of each transaction, goods transferred during thetransaction, services rendered during the transaction, and the like. Insome embodiments, organization computing system 104 may also base theprediction on transaction information associated with other customers.For example, handler 118 may provide as input to machine learning module125 one or more items of transaction information 134 associated withother customer profiles 126.

In some embodiments, method 300 may include step 313. At step 313,organization computing system 104 may interface with facility web server154 to suggest a predicted transaction to the customer. For example,organization computing system 104 may instruct web server 154 to promptapplication 155 to push a notification to the customer. The notificationmay include a suggested transaction, based on the transaction history ofthe user. Thus, the customer may be able to confirm the details of thetransaction before organization computing system 104 instructs facility102 to begin preparing the transaction. In some embodiments, if, afterreviewing the notification, the customer does not agree with thepredicted transaction, customer may request that facility web server 154provide another suggestion or, in some cases, forego generating aprediction. Accordingly, facility web server 154 may relay theinstructions from the customer to organization computing system 104. Insome embodiments, if, after reviewing the notification, the customeragrees with the predicted transaction, customer may approve of thepredicted order via application 155. Facility web server 154 may, inturn, relay the approval to organization computing system 104.

For the remainder of the operations discussed in FIG. 3, assume that thecustomer approves of the suggested, predicted order.

At step 314, organization computing system 104 may transmit thepredicted transaction to facility 102. For example, organizationcomputing system 104 may transmit the predicted transaction to facility102 via controller 114. Transmitting the predicted transaction tofacility 102 may allow employees of facility 102 to being preparing thetransaction. In some embodiments, facility 102 may choose to confirm thetransaction with the customer prior to preparation. In some embodiments,facility 102 may choose to confirm the customer is within facility 102for a predetermined threshold of time, before beginning transactionpreparation. For example, controller 114 of facility 102 may pingorganization computing system 104. requesting an update as to whetherthe customer is still with facility 102.

At step 316, organization computing system 104 may determine whether thecustomer has pre-authorized an automatic payment feature. For example,organization computing system 104 may determine whether the customer haslinked a financial account their customer profile, thereby authorizingpayment therefrom.

If, at step 316, organization computing system 104 determines that thecustomer has pre-authorized an automatic payment feature, then method300 ends, and the customer may pay at POS terminal 110. If, however, atstep 316, organization computing system 104 determines that the customerhas pre-authorized an automatic payment feature, then method 300proceeds to step 318.

At step 318, upon determining that the customer has pre-authorized theautomatic payment feature, organization computing system 104 may submita payment from the pre-authorized account to facility 102.

Accordingly, facility 102 is able to begin preparation of a potentialtransaction with a given customer given the customer's transactionhistory at facility 102. Further, facility 102 is able to receive anautomatic payment from organization computing system 104, whenpreviously authorized by the customer. Such process improves theefficiency and customer throughput of facility 102, while simultaneouslysimplifying the transaction process for customers.

FIG. 4 is a block diagram illustrating an exemplary computingenvironment 400, according to some embodiments. Computing environment400 includes computing system 402 and computing system 452. Computingsystem 402 may be representative of facility 102. Computing system 452may be representative of organization computing system 104.

Computing system 402 may include a processor 404, a memory 406, astorage 408, and a network interface 410. In some embodiments, computingsystem 402 may be coupled to one or more I/O device(s) 412 (e.g.,keyboard, mouse, etc.). Such I/O devices 412 may include, for example,one or more cameras 414 and one or more POS terminals 415.

One or more cameras 414 may be positioned within a facility. Forexample, one or more cameras 414 may be positioned to capture customersat each POS terminal 415. In some embodiments, at least one camera ofthe one or more cameras 414 may be positioned exterior to facility orlocated inside the facility, such that the camera captures customersentering and leaving the facility. Each POS terminal 415 may berepresentative of a general purpose computing device programmed toreceive transactions from one or more customers and facilitate paymentbetween the one or more customers and facility.

Processor 404 may retrieve and execute program code 420 (i.e.,programming instructions) stored in memory 406, as well as stores andretrieves application data. Processor 404 may be included to berepresentative of a single processor, multiple processors, a singleprocessor having multiple processing cores, and the like. Networkinterface 410 may be any type of network communications allowingcomputing system 402 to communicate externally via computing network405. For example, network interface 410 is configured to enable externalcommunication with computing system 452.

Storage 408 may be, for example, a disk storage device. Although shownas a single unit, storage 408 may be a combination of fixed and/orremovable storage devices, such as fixed disk drives, removable memorycards, optical storage, network attached storage (NAS), storage areanetwork (SAN), and the like.

Memory 406 may include application 416, operating system 418, andprogram code 420. Program code 420 may be accessed by processor 404 forprocessing (i.e., executing program instructions). Program code 420 mayinclude, for example, executable instructions for communicating withcomputing system 452 to display one or more pages of website 464.Application 416 may enable a customer of computing system 402 to accessa functionality of computing system 452. For example, application 416may access content managed by computing system 452, such as website 464.The content that is displayed to a customer of computing system 402 maybe transmitted from computing system 452 to computing system 402, andsubsequently processed by application 416 for display through agraphical user interface (GUI) of computing system 402.

Computing system 452 may include a processor 454, a memory 456, astorage 458, and a network interface 460. In some embodiments, computingsystem 452 may be coupled to one or more I/O device(s) 462. In someembodiments, computing system 452 may be in communication with database106.

Processor 454 may retrieve and execute program code 468 (i.e.,programming instructions) stored in memory 456, as well as stores andretrieves application data. Processor 454 is included to berepresentative of a single processor, multiple processors, a singleprocessor having multiple processing cores, and the like. Networkinterface 460 may be any type of network communications enablingcomputing system 452 to communicate externally via computing network405. For example, network interface 460 allows computing system 452 tocommunicate with computer system 402.

Storage 458 may be, for example, a disk storage device. Although shownas a single unit, storage 458 may be a combination of fixed and/orremovable storage devices, such as fixed disk drives, removable memorycards, optical storage, network attached storage (NAS), storage areanetwork (SAN), and the like.

Memory 456 may include website 464, operating system 466, program code468, handler 470, and transaction agent 472. Program code 468 may beaccessed by processor 454 for processing (i.e., executing programinstructions). Program code 468 may include, for example, executableinstructions configured to perform steps discussed above in conjunctionwith FIG. 3. As an example, processor 454 may access program code 468 toperform operations for generating one or more rules exempt from atransaction hold. In another example, processor 454 may access programcode 468 to perform operations for customizing a customer experience.Website 464 may be accessed by computing system 402. For example,website 464 may include content accessed by computing system 402 via aweb browser or application.

Handler 470 may be configured to manage information stored in database106. For example, handler 470 may be configured to update clientinformation in database 106 when prompted by a user, when prompted byemployees of the facility, or when prompted by another component ofcomputing system 452.

Transaction agent 472 may be configured to predict a customertransaction prior to a customer arriving at, for example, a POS terminal415. In some embodiments, transaction agent 472 may be configured toidentify one or more users (or customers) using one or more videostreams received from one or more cameras 414 positioned therein. Forexample, transaction agent 472 may be configured to parse through one ormore video streams in real-time, to identify one or more customersarriving at (or already inside) facility, based on previously identifiedfacial features stored in database 106. For example, in someembodiments, transaction agent 472 may match a customer's face to acustomer in database 106 using three dimensional facial recognitionmethodologies, skin texture analysis methodology, or a combination ofthe same.

In some embodiments, transaction agent 472 may be configured totranslate audible utterances of a customer to text for further analysis.In some embodiments, each POS terminal 415 may include a microphone tocapture communications between a customer and an employee operation POSterminal 415. Transaction agent 472 may include software configured toprovide text-based descriptions of audible input. Transaction agent 472may parse the text-based description to identify one or more details ofa customer's transaction at the facility. Transaction agent 472 mayinclude a machine learning module configured to train a prediction modelused by transaction agent 472 to predict details of a customer's futuretransaction. To train the prediction model, machine learning module mayreceive, as input, one or more streams of customer activity. The one ormore streams of customer activity may correspond to previous transactionat given facilities. Such streams of activity may include the facility,the time of day, the day of the week, items purchased during thetransaction, and the like. In some embodiments, machine learning modulemay further receiver, as input, one or more streams of activityassociated with additional customers. As such, machine learning modulemay leverage both customer specific and customer agnostic information topredict details of a customer's transaction when, for example, thecustomer arrives at the facility.

While the foregoing is directed to embodiments described herein, otherand further embodiments may be devised without departing from the basicscope thereof. For example, aspects of the present disclosure may beimplemented in hardware or software or a combination of hardware andsoftware. One embodiment described herein may be implemented as aprogram product for use with a computer system. The program(s) of theprogram product define functions of the embodiments (including themethods described herein) and can be contained on a variety ofcomputer-readable storage media. Illustrative computer-readable storagemedia include, but are not limited to: (i) non-writable storage media(e.g., read-only memory (ROM) devices within a computer, such as CD-ROMdisks readably by a CD-ROM drive, flash memory, ROM chips, or any typeof solid-state non-volatile memory) on which information is permanentlystored; and (ii) writable storage media (e.g., floppy disks within adiskette drive or hard-disk drive or any type of solid staterandom-access memory) on which alterable information is stored. Suchcomputer-readable storage media, when carrying computer-readableinstructions that direct the functions of the disclosed embodiments, areembodiments of the present disclosure.

It will be appreciated to those skilled in the art that the precedingexamples are exemplary and not limiting. It is intended that allpermutations, enhancements, equivalents, and improvements thereto areapparent to those skilled in the art upon a reading of the specificationand a study of the drawings are included within the true spirit andscope of the present disclosure. It is therefore intended that thefollowing appended claims include all such modifications, permutations,and equivalents as fall within the true spirit and scope of theseteachings.

What is claimed:
 1. A method of customizing a customer experience,comprising: generating, by a computing system, a prediction model forpredicting customer transactions by learning, via one or more machinelearning models, the customer's spending habits at the facility;receiving one or more first video streams captured by a first camerapositioned at a first location proximate to a facility, the one or morefirst video streams comprising at least one customer; identifying anidentity of the customer by parsing the one or more first video streamsto identify one or more audio or visual cues of the customer;predicting, using the prediction model, a new transaction at thefacility; confirming that the customer remains in the facility byidentifying the customer in one or more second video streams captured bya second camera positioned at a second location proximate to thefacility; and in response to confirming that the customer remains in thefacility, notifying the computing device positioned in the facility inpreparation of the new transaction.
 2. The method of claim 1, furthercomprising: determining that the customer has pre-authorized paymentfrom a customer account; and notifying the computing device positionedin the facility of the pre-authorized payment.
 3. The method of claim 1,wherein generating, by the computing system, the prediction model forpredicting customer transactions by learning, via the one or moremachine learning models, the customer's spending habits at the facilitycomprises: identifying one or more previous transactions at the facilityto learn a transaction pattern at the facility.
 4. The method of claim3, further comprising: identifying a second one or more previoustransactions at other facilities to learn a second transaction patternamong the other facilities.
 5. The method of claim 3, furthercomprising: for each previous transaction of the one or more previoustransactions at the facility, identifying a day of the week and time ofday each previous transaction occurred.
 6. The method of claim 1,wherein predicting, using the prediction model, the new transaction atthe facility comprises: identifying a current day of the week;identifying a current time of the current day; and generating aprediction based on the current day of the week and the current time ofthe current day.
 7. The method of claim 1, wherein the second locationis proximate a point-of-sale terminal.
 8. A system, comprising: aprocessor; and a memory having programming instructions stored thereon,which, when executed by the processor, performs one or more operationscomprising: generating a prediction model for predicting customertransactions by learning, via one or more machine learning models, thecustomer's spending habits at the facility; receiving one or more firstvideo streams captured by a first camera positioned at a first locationproximate to a facility, the one or more first video streams comprisingat least one customer; identifying an identity of the customer byparsing the one or more first video streams to identify one or moreaudio or visual cues of the customer; predicting, using the predictionmodel, a new transaction at the facility; confirming that the customerremains in the facility by identifying the customer in one or moresecond video streams captured by a second camera positioned at a secondlocation proximate to the facility; and in response to confirming thatthe customer remains in the facility, notifying the computing devicepositioned in the facility in preparation of the new transaction.
 9. Thesystem of claim 8, wherein the one or more operations further comprise:determining that the customer has pre-authorized payment from a customeraccount; and notifying the computing device positioned in the facilityof the pre-authorized payment.
 10. The system of claim 8, whereingenerating the prediction model for predicting customer transactions bylearning, via the one or more machine learning models, the customer'sspending habits at the facility comprises: identifying one or moreprevious transactions at the facility to learn a transaction pattern atthe facility.
 11. The system of claim 10, further comprising:identifying a second one or more previous transactions at otherfacilities to learn a second transaction pattern among the otherfacilities.
 12. The system of claim 10, further comprising: for eachprevious transaction of the one or more previous transactions at thefacility, identifying a day of the week and time of day each previoustransaction occurred.
 13. The system of claim 10, wherein predicting,using the prediction model, the new transaction at the facilitycomprises: identifying a current day of the week; identifying a currenttime of the current day; and generating a prediction based on thecurrent day of the week and the current time of the current day.
 14. Thesystem of claim 10, wherein the second location is proximate apoint-of-sale terminal.
 15. A non-transitory computer readable mediumincluding one or more instructions which, when executed by one or moreprocessors, cause the one or more processors to perform operationscomprising: generating, by a computing system, a prediction model forpredicting customer transactions by learning, via one or more machinelearning models, the customer's spending habits at the facility;receiving one or more first video streams captured by a first camerapositioned at a first location proximate to a facility, the one or morefirst video streams comprising at least one customer; identifying anidentity of the customer by parsing the one or more first video streamsto identify one or more audio or visual cues of the customer;predicting, using the prediction model, a new transaction at thefacility; confirming that the customer remains in the facility byidentifying the customer in one or more second video streams captured bya second camera positioned at a second location proximate to thefacility; and in response to confirming that the customer remains in thefacility, notifying the computing device positioned in the facility inpreparation of the new transaction.
 16. The non-transitory computerreadable medium of claim 15, further comprising: determining that thecustomer has pre-authorized payment from a customer account; andnotifying the computing device positioned in the facility of thepre-authorized payment.
 17. The non-transitory computer readable mediumof claim 15, wherein generating, by the computing system, the predictionmodel for predicting customer transactions by learning, via the one ormore machine learning models, the customer's spending habits at thefacility comprises: identifying one or more previous transactions at thefacility to learn a transaction pattern at the facility.
 18. Thenon-transitory computer readable medium of claim 17, further comprising:identifying a second one or more previous transactions at otherfacilities to learn a second transaction pattern among the otherfacilities.
 19. The non-transitory computer readable medium of claim 17,further comprising: for each previous transaction of the one or moreprevious transactions at the facility, identifying a day of the week andtime of day each previous transaction occurred.
 20. The non-transitorycomputer readable medium of claim 15, wherein predicting, using theprediction model, the new transaction at the facility comprises:identifying a current day of the week; identifying a current time of thecurrent day; and generating a prediction based on the current day of theweek and the current time of the current day.